Reducing Adversarial Vulnerability through Adaptive Training Batch Size

Neural networks possess an ability to generalize well data distribution, extent that they are capable of fitting a randomly labeled data. But also known be extremely sensitive adversarial examples. Batch Normalization (BatchNorm), very commonly part deep learning architecture, has been found increase vulnerability. Fixup Initialization (Fixup Init) shown as alternative BatchNorm, which can considerably strengthen the against This robustness improved further by employing smaller batch size in training. The latter, however, comes with tradeoff form significant training time (up ten times longer when reducing from default 128 8 for ResNet-56). In this paper, we propose workaround problem starting small and gradually it larger ones during We empirically show our proposal still improve 5.73\%) ResNet-56 Init 128. At same time, keeps shorter (only 4 longer, instead 10 times).